86 research outputs found
Enhancing Network Initialization for Medical AI Models Using Large-Scale, Unlabeled Natural Images
Pre-training datasets, like ImageNet, have become the gold standard in
medical image analysis. However, the emergence of self-supervised learning
(SSL), which leverages unlabeled data to learn robust features, presents an
opportunity to bypass the intensive labeling process. In this study, we
explored if SSL for pre-training on non-medical images can be applied to chest
radiographs and how it compares to supervised pre-training on non-medical
images and on medical images. We utilized a vision transformer and initialized
its weights based on (i) SSL pre-training on natural images (DINOv2), (ii) SL
pre-training on natural images (ImageNet dataset), and (iii) SL pre-training on
chest radiographs from the MIMIC-CXR database. We tested our approach on over
800,000 chest radiographs from six large global datasets, diagnosing more than
20 different imaging findings. Our SSL pre-training on curated images not only
outperformed ImageNet-based pre-training (P<0.001 for all datasets) but, in
certain cases, also exceeded SL on the MIMIC-CXR dataset. Our findings suggest
that selecting the right pre-training strategy, especially with SSL, can be
pivotal for improving artificial intelligence (AI)'s diagnostic accuracy in
medical imaging. By demonstrating the promise of SSL in chest radiograph
analysis, we underline a transformative shift towards more efficient and
accurate AI models in medical imaging
Empowering Clinicians and Democratizing Data Science: Large Language Models Automate Machine Learning for Clinical Studies
A knowledge gap persists between Machine Learning (ML) developers (e.g., data
scientists) and practitioners (e.g., clinicians), hampering the full
utilization of ML for clinical data analysis. We investigated the potential of
the chatGPT Advanced Data Analysis (ADA), an extension of GPT-4, to bridge this
gap and perform ML analyses efficiently. Real-world clinical datasets and study
details from large trials across various medical specialties were presented to
chatGPT ADA without specific guidance. ChatGPT ADA autonomously developed
state-of-the-art ML models based on the original study's training data to
predict clinical outcomes such as cancer development, cancer progression,
disease complications, or biomarkers such as pathogenic gene sequences.
Strikingly, these ML models matched or outperformed their published
counterparts. We conclude that chatGPT ADA offers a promising avenue to
democratize ML in medicine, making advanced analytics accessible to non-ML
experts and promoting broader applications in medical research and practice
What Does DALL-E 2 Know About Radiology?
Generative models, such as DALL-E 2 (OpenAI), could represent promising future tools for image generation, augmentation, and manipulation for artificial intelligence research in radiology, provided that these models have sufficient medical domain knowledge. Herein, we show that DALL-E 2 has learned relevant representations of x-ray images, with promising capabilities in terms of zero-shot text-to-image generation of new images, the continuation of an image beyond its original boundaries, and the removal of elements; however, its capabilities for the generation of images with pathological abnormalities (eg, tumors, fractures, and inflammation) or computed tomography, magnetic resonance imaging, or ultrasound images are still limited. The use of generative models for augmenting and generating radiological data thus seems feasible, even if the further fine-tuning and adaptation of these models to their respective domains are required first
Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels
Artificial intelligence (AI) methods are revolutionizing medical image
analysis. However, robust AI models require large multi-site datasets for
training. While multiple stakeholders have provided publicly available
datasets, the ways in which these data are labeled differ widely. For example,
one dataset of chest radiographs might contain labels denoting the presence of
metastases in the lung, while another dataset of chest radiograph might focus
on the presence of pneumonia. With conventional approaches, these data cannot
be used together to train a single AI model. We propose a new framework that we
call flexible federated learning (FFL) for collaborative training on such data.
Using publicly available data of 695,000 chest radiographs from five
institutions - each with differing labels - we demonstrate that large and
heterogeneously labeled datasets can be used to train one big AI model with
this framework. We find that models trained with FFL are superior to models
that are trained on matching annotations only. This may pave the way for
training of truly large-scale AI models that make efficient use of all existing
data.Comment: 2 figures, 3 tables, 5 supplementary table
Time-efficient combined morphologic and quantitative joint MRI based on clinical image contrasts -- An exploratory in-situ study of standardized cartilage defects
OBJECTIVES: Quantitative MRI techniques such as T2 and T1 mapping are
beneficial in evaluating cartilage and meniscus. We aimed to evaluate the
MIXTURE (Multi-Interleaved X-prepared Turbo-Spin Echo with IntUitive
RElaxometry) sequences that provide morphologic images with clinical turbo
spin-echo (TSE) contrasts and additional parameter maps versus reference TSE
sequences in an in-situ model of human cartilage defects.
MATERIALS AND METHODS: Prospectively, standardized cartilage defects of 8mm,
5mm, and 3mm diameter were created in the lateral femora of 10 human cadaveric
knee specimens (8110 years, nine male/one female). Using a clinical 3T MRI
scanner and knee coil, MIXTURE sequences combining (i) proton-density weighted
fat-saturated (PD-w FS) images and T2 maps and (ii) T1-weighted images and
T1 maps were acquired before and after defect creation, alongside the
corresponding 2D TSE and 3D TSE reference sequences. Defect delineability, bone
texture, and cartilage relaxation times were quantified. Inter-sequence
comparisons were made using appropriate parametric and non-parametric tests.
RESULTS: Overall, defect delineability and texture features were not
significantly different between the MIXTURE and reference sequences. After
defect creation, relaxation times increased significantly in the central femur
(for T2) and all regions combined (for T1).
CONCLUSION: MIXTURE sequences permit time-efficient simultaneous morphologic
and quantitative joint assessment based on clinical image contrasts. While
providing T2 or T1 maps in clinically feasible scan time, morphologic
image features, i.e., cartilage defect delineability and bone texture, were
comparable between MIXTURE and corresponding reference sequences.Comment: 12 pages (main body), 3 tables, 6 figure
Adversarial attacks and adversarial robustness in computational pathology.
Artificial Intelligence (AI) can support diagnostic workflows in oncology by aiding diagnosis and providing biomarkers directly from routine pathology slides. However, AI applications are vulnerable to adversarial attacks. Hence, it is essential to quantify and mitigate this risk before widespread clinical use. Here, we show that convolutional neural networks (CNNs) are highly susceptible to white- and black-box adversarial attacks in clinically relevant weakly-supervised classification tasks. Adversarially robust training and dual batch normalization (DBN) are possible mitigation strategies but require precise knowledge of the type of attack used in the inference. We demonstrate that vision transformers (ViTs) perform equally well compared to CNNs at baseline, but are orders of magnitude more robust to white- and black-box attacks. At a mechanistic level, we show that this is associated with a more robust latent representation of clinically relevant categories in ViTs compared to CNNs. Our results are in line with previous theoretical studies and provide empirical evidence that ViTs are robust learners in computational pathology. This implies that large-scale rollout of AI models in computational pathology should rely on ViTs rather than CNN-based classifiers to provide inherent protection against perturbation of the input data, especially adversarial attacks
Two for One -- Combined Morphologic and Quantitative Knee Joint MRI Using a Versatile Turbo Spin-Echo Platform
Introduction: Quantitative MRI techniques such as T2 and T1\r{ho} mapping are
beneficial in evaluating knee joint pathologies; however, long acquisition
times limit their clinical adoption. MIXTURE (Multi-Interleaved X-prepared
Turbo-Spin Echo with IntUitive RElaxometry) provides a versatile turbo
spin-echo (TSE) sequence platform for simultaneous morphologic and quantitative
joint imaging yet lacks comparative evaluation in basic and translational
research contexts.
Methods: Two MIXTURE sequences were designed along clinical requirements: (i)
MIX1, combining proton density (PD)-weighted fat-saturated (FS) images and
quantitative T2 mapping (acquisition time: 4:59 min), and (ii) MIX2, combining
T1-weighted images with quantitative T1\r{ho} mapping (6:38 min). MIXTURE
sequences and their reference 2D and 3D TSE counterparts were acquired from ten
human cadaveric knee joints using a clinical 3T MRI scanner and knee coil.
Contrast, contrast-to-noise ratios, and coefficients of variation were
comparatively evaluated using parametric tests. Clinical radiologists (n=3)
assessed diagnostic quality as a function of sequence and anatomic structure
using 5-point Likert scales and ordinal regression. The significance level was
set to {\alpha}=0.01.
Results: MIX1 and MIX2 had at least equal diagnostic quality compared to the
2D and 3D TSE sequences of the same image weighting. Contrast,
contrast-to-noise ratios, and coefficients of variation were largely similar
for the PD-weighted FS and T1-weighted images.
Discussion: In clinically feasible scan times, the MIXTURE sequence platform
yields (i) morphologic images of diagnostic quality and adjustable TSE-based
contrasts and (ii) quantitative parameter mapping with additional insights on
soft tissue composition and ultrastructure.Comment: 13 pages (main text), 7 figures, 3 table
- …